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ABSTRACT 

An  immersive  technology  being  developed  by  the 
Army  to  provide  enhanced  situation  awareness  for 
military  personnel  involves  the  use  of  tactile  stimulators 
arranged  in  a  matrix  pattern  surrounding  the  torso.  When 
activated  sequentially  the  tactile  display  can  be  swept 
over  the  torso  to  provide  a  sense  of  motion.  This  tactile 
stimulus  can  alter  the  perceived  experience  of  self-motion 
produced  by  large  field-of-view  displays.  The  tactile 
display  can  also  alter  the  reflex  responses  to  visual 
stimuli. 


1.  INTRODUCTION 

Spatial  disorientation  (SD)  and  the  subsequent  loss  of 
situation  awareness  (SA)  account  for  a  significant 
percentage  of  mishaps  in  aviation.  The  safety  of  the 
aircraft  and  the  ability  to  perform  the  aircraft’s  mission  is 
highly  dependent  on  the  pilot  having  an  accurate 
awareness  of  the  current  situation,  including  the  state  of 
one’s  own  aircraft,  mission  goals,  external  conditions, 
other  aircraft,  and  other  hostile  factors.  Without  this  SA, 
the  pilot  will  be  unable  to  effectively  perform  the  mission. 
Although  not  restricted  to  the  aviation  environment,  when 
loss  of  SA  occurs  in  aviation,  the  consequences  are  more 
severe,  frequently  resulting  in  loss  of  life  and  or  aircraft. 
Acquiring  and  maintaining  SA  becomes  increasingly 
more  difficult  as  the  complexity  and  dynamics  of  the 
military  aviation  environment  increase. 

Situation  awareness  has  been  defined  in  the  following 
manner,  “The  perception  of  the  elements  in  the 
environment  within  a  volume  of  space  and  time,  the 
comprehension  of  their  meaning  and  the  projection  of 
their  status  in  the  near  future”  (Endsley,  1988,  1995). 

The  first  and  critical  step  in  acquiring  and 
maintaining  SA  is  to  perceive  the  status,  attributes,  and 
dynamics  of  elements  in  the  environment  (SA  Level  1, 
Endsley,  1995).  Without  an  accurate  percept  of  SA  Level 
1,  the  pilot  will  experience  a  loss  of  SA  that  can  have 
severe  and  costly  consequences.  Spatial  disorientation 
occurs  when  the  pilot  has  an  incorrect  perception  of  the 


attitude,  altitude,  or  motion  of  one’s  own  aircraft  relative 
to  the  earth  or  other  significant  objects.  This  corresponds 
to  an  inaccurate  perception  of  the  elements  in  the  current 
situation  (SA  Level  1). 

Surveys  spanning  the  past  40  years  have  shown  that 
SD  mishaps  have  a  critical  and  often  severe  outcome. 
Based  on  accident  rates  for  the  United  States  (U.S.)  Air 
Force,  Navy,  and  Army,  spatial  disorientation  mishaps 
result  in  the  tragic  loss  of  40  lives  on  average  per  year 
(Gillingham,  1992;  Durnford,  Crowley  et  ah,  1995; 
Braithwaite,  Groh,  and  Alvarez,  1997).  The  cost  of 
spatial  disorientation  mishaps  also  includes  mission 
failure,  the  impairment  of  mission  effectiveness,  and  the 
monetary  value  of  aircraft  and  equipment  loss. 
Considering  the  number  of  military  air  forces,  commercial 
and  general  aviation,  the  estimated  annual  material  cost  of 
spatial  disorientation  mishaps  is  in  the  billions  of  dollars 
(Gillingham,  1992).  Aviators  need  improved  tools  to 
recognize  and/or  prevent  spatial  disorientation. 

Since  2001  the  number  of  aviation  mishaps  attributed 
to  degraded  visual  environment  mishaps  has  increased 
dramatically  due  to  increased  operational  tempo  in  desert 
environments.  An  illusion  associated  with  brownout 
conditions  is  vection  -  the  compelling  sense  of  self 
motion  in  physically  stationary  observers.  Visually 
induced  vection  is  frequently  accompanied  by  behavioral 
and  reflex  responses  including  postural  adjustments, 
motion/simulator  sickness  and  optokinetic  nystagmus. 

The  latter  is  a  visual  reflex  consisting  of  fast  and  slow 
phase  eye  movements  to  maintain  focus  on  objects  in  the 
moving  visual  field. 

In  1990,  Rupert,  Mateczun,  and  Guedry  (1990) 
proposed  that  a  tactile  interface  could  be  used  as  a  “more 
natural”  approach  to  convey  position  and  motion 
perception  during  flight.  Since  tactile  orientation  cues 
present  fast  robust  responses  while  posing  minimal 
demands  on  cognitive  reserve,  the  touch  sensation  is  an 
ideal  candidate  to  provide  continuous,  intuitive  veridical 
orientation  information. 
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Fig.  1.  Tactile  Situation  Awareness  System  (TSAS) 
concept 

The  Tactile  Situational  Awareness  System  (TSAS), 
developed  initially  by  National  Aeronautics  and  Space 
Administration  (NASA)  and  the  Naval  Aerospace 
Medical  Research  Laboratory,  and  currently  continuing 
development  with  the  U.S.  Army  Aeromedical  Research 
Laboratory  (USAARL),  was  designed  to  improve 
situational  awareness  by  presenting  information  to 
military  personnel  via  the  sensation  of  touch.  The  TSAS 
system  consists  of  a  controller  and  miniature  tactile 
stimulators  called  “factors.”  The  TSAS  can  obtain  aircraft 
position,  velocity,  altitude,  and  threat  information  from 
the  onboard  aircraft  data  bus.  Appropriate  information,  as 
dictated  by  the  TSAS  mode  and  processing  algorithm,  is 
then  displayed  to  the  pilot  using  the  factors.  The  factors 
are  embedded  in  an  air-cooled  cooling  vest  originally 
developed  by  DRDC  Toronto.  The  above  illustration  (Fig. 

1)  indicates  how  the  matrix  of  columns  and  rows  of 
factors  can  be  arranged  on  the  torso  of  the  body.  These 
factors  may  be  mapped  to  points  in  the  external 
environment  as  indicated  by  the  globe  surrounding  the 
subject  in  the  above  diagram. 

The  TSAS  has  been  successfully  demonstrated  to 
present  flight  information  in  fixed  and  rotary  wing  aircraft 
(Griffin,  Pera,  Cabrera,  and  Moore,  2001;  McGrath,  2000; 
Rupert,  2000).  The  TSAS  has  previously  been  evaluated 
for  its  robustness,  intuitiveness,  and  effectiveness  during 
general  experimental  tasks  in  the  CV-22,  MF1-53M,  and 
the  MF1-60K  simulators,  and  in  the  T-34C,  UF1-60A  (Fig. 

2) ,  and  MF1-53M  aircraft. 

Initial  fixed-wing  flight  tests  provided  pilots  intuitive 
non-visual  information  as  to  the  direction  “down.”  The 
most  difficult  task  for  rotary  wing  pilots  is  hovering  under 
visions  of  restricted  visibility.  Several  algorithms  were 
developed  to  provide  velocity  information  for  helicopter 
pilots.  In  the  process  of  developing  a  velocity  cue,  the 
investigators  experimented  with  providing  flow 
information  on  the  torso  of  the  body. 

The  TSAS  technologies  have  shown  the  potential  to 
increase  pilot  SA  and  reduce  pilot  workload,  especially 


during  complex  flight  conditions.  Using  TSAS,  pilots 
demonstrated  enhanced  control  of  hover  maneuvers, 
including  transitions  to  and  from  forward  flight  in 
degraded  visual  conditions,  relying  on  tactile  cues  for  the 
necessary  information.  The  awareness  of  aircraft 
movement  over  the  ground  or  “drift”  without  looking  at  a 
visual  instrument  was  the  most  important  feature  of  TSAS 
(Cheung,  Rupert  et  al.,  2004;  Jennings,  Schultz  et  ah, 
2004). 

The  tactile  display  provided  the  opportunity  to  devote 
more  time  to  other  instruments  and  systems  when  flying 
in  task  saturated  conditions.  The  TSAS  permitted  the  pilot 
to  concentrate  on  mission  tasks,  thereby  reducing 
workload.  These  effects  can  substantially  increase 
mission  effectiveness. 

The  awareness  of  aircraft  velocity  over  the  ground  or 
“drift”  without  looking  at  a  visual  instrument  was  the 
biggest  advantage  of  TSAS.  The  maintenance  of  SA 
during  reduced  visual  conditions  was  enhanced.  The 
TSAS  permitted  the  pilot  to  concentrate  on  mission  tasks, 
thereby  reducing  cognitive  stress.  Overall,  TSAS 
decreased  pilot  workload,  enhanced  SA,  and  increased  the 
potential  for  survivability  and  lethality  (McGrath  1998). 


Fig.  2.  USAARL  UF1-60  research  aircraft 

The  current  study  examines  whether  tactile  displays 
can  reduce  visual  illusions  encountered  in  the  aerospace 
environment. 


2.  METHODS 
2.1  Vection  Stimulus 

Self-motion  perception  was  provoked  in  12  healthy 
volunteers  sitting  motionless  within  the  Visual  Vestibular 
Sphere  Device  (VVSD)  (Fig.  3  and  Fig.  4).  The  VVSD  is 
a  12  foot  diameter  sphere  that  can  be  rotated  around  a 
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subject  seated  upright  with  his/her  head  located  in  the 
center  of  sphere  rotation. 


Fig.  3  Visual  Vestibular  Sphere  Device  (VVSD)  exterior 
view 


Following  45  seconds  of  visual  vection  stimulus  at 
constant  velocity,  the  tactile  stimulus  was  applied  for  30 
seconds. 


3.  RESULTS 

All  subjects  reported  self-motion  opposite  to  the 
direction  of  sphere  motion  within  15  seconds  of  onset  of 
sphere  rotation.  There  was  no  difference  between 
clockwise  and  counterclockwise  rotation.  All  subjects 
reported  “saturated”  vection  -  the  rotating  sphere 
appeared  motionless  with  all  motion  perceived  as  self- 
motion. 

When  the  tactile  stimulus  was  applied  all  subjects 
experienced  a  change  in  velocity  of  the  vection  (self- 
motion)  experience.  All  subjects  noted  a  decrease  in 
perceived  velocity.  Two  subjects  reported  that  the  sphere 
appeared  to  stop  momentarily  and  then  continued  to  rotate 
at  a  slower  velocity.  Three  subjects  noted  that  the  tactile 
stimulus  was  “jerky”. 


Optokinetic  (OKN)  eye  movement  data  was  available 
for  1 1  subjects.  When  the  tactile  flow  stimulus  was 
applied  during  constant  sphere  velocity,  six  subjects 
(55%)  demonstrated  a  decrease  in  magnitude  (intensity) 
of  slow  phase  velocity  or  change  in  frequency  of  OKN  as 
seen  in  the  following  eye  movement  recordings  (Fig.  5 
and  Fig.  6).  The  changes  in  nystagmus  were 
accompanied  by  a  decrease  in  perceived  velocity  of 
rotation. 

Five  of  the  subjects  who  reported  a  decrease  in 
vection  velocity  did  not  demonstrate  a  clear  change  in 
slow  phase  velocity  or  frequency  on  the  OKN  recordings. 


Fig.  4.  VVSD  interior  view  during  rotation 

When  rotated  around  the  stationary  subject,  the 
random  dot  pattern  which  fills  the  subject’s  entire  field- 
of-view  provides  a  strong  vection  stimulus.  Florizontal 
eye  movements  were  monitored  with  electrooculography 
(EOG).  The  sphere  rotation  velocity  was  varied  between 
15  and  60  degrees  per  second. 

2.2  Tactile  Stimulus 

The  torso  garment  consisted  of  8  columns  with  5 
pneumatic  tactile  stimulators  (tactors)  in  each  column. 
Tactor  columns  were  located  on  the  front,  back,  left  and 
right  sides  and  at  45  degrees  between  these  cardinal 
headings.  The  rotating  tactile  stimulus  consisted  of  two 
adjacent  tactor  columns  being  activated  simultaneously 
and  advanced  clockwise  or  counterclockwise  around  the 
torso.  Stimulus  frequency  was  40  Hz  . 
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Subject  20 


Fig.  5.  Change  in  slow  phase  velocity  following  application  of  tactile  flow  stimulus.  Slope  of  line  indicates  velocity  of 
slow  phase  of  OKN  nystagmus.  Shortly  after  the  application  of  the  tactile  flow  stimulus  (vertical  green  bar)  the  slow  phase 
velocity  decreased. 


Subject  24 


Time  (sec) 


Fig.  6.  Change  in  frequency  of  optokinetic  nystagmus  following  application  of  tactile  flow  stimulus.  Following  application 
of  the  tactile  flow  stimulus  (vertical  green  bar)  this  subject  demonstrates  an  increased  frequency  of  nystagmus. 


4.  DISCUSSION 

For  institutional  review  board  reasons,  all  subjects  in 
this  experiment  were  informed  prior  to  the  experiment 
that  the  chair  would  not  rotate.  Despite  full  knowledge  of 
the  absence  of  motion,  no  subject  could  suppress  the 
illusory  sense  of  circular  vection. 

The  experience  of  tactile  “jerkiness”  by  three  subjects 
relates  to  the  stimulus  area  and  density.  In  the  current 
experiment  the  stimulus  was  applied  to  two  columns  at  a 
time  instead  of  several  columns.  When  the  density  of  the 
array  is  increased  with  additional  columns  there  is  also 
less  tendency  to  experience  the  stimulus  “jumping”  from 
column  to  column  in  a  saltatory  fashion. 


There  has  been  considerable  debate  in  the  aviation 
industry  as  to  whether  motion  based  simulators  provide 
better  training  than  non-motion  based  simulators.  Motion 
based  simulators  have  many  drawbacks  in  addition  to 
significantly  higher  costs  associated  with  the  initial 
procurement  and  maintenance.  The  large  footprint  and 
non-portability  prevents  deployment  of  motion  based 
simulators  with  troops  to  the  war  zone  where  simulators 
can  best  provide  mission  rehearsal  capabilities.  Small 
non-motion  simulators  are  promoted  for  the  capability  of 
deployment  but  detractors  claim  that  realism  suffers  from 
lack  of  motion  cues. 

The  ability  of  tactile  stimuli  to  alter  or  provide 
sensations  of  motion  may  promote  the  use  of  non-motion, 
portable  simulators  in-theater  as  mission  rehearsal 
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devices.  Considerable  research  needs  to  be  carried  out  to 
determine  the  optimal  hardware  configurations  as  well  as 
to  develop  software  programs  that  provide  compelling 
sensations. 

There  are  several  circumstances  that  produce  vection 
illusions  in  the  aerospace  environment,  often  leading  to 
mishaps.  The  issue  of  blowing  sand  or  snow  has  led  pilots 
to  experience  drift  in  the  opposite  direction  of  the  blowing 
particles.  There  have  been  several  mishaps  where  pilots 
have  made  control  inputs  to  “correct”  for  the  apparent 
motion  with  disastrous  results. 

In  a  debrief  following  the  first  moon  landing  by  Neil 
Armstrong  and  Buzz  Aldrin,  the  astronauts  discussed  the 
visual  obscuration  posed  by  the  surface  particulate  matter 
being  driven  across  the  surface  of  the  moon  from  the 
force  of  the  retro  rockets  as  they  approached  the  lunar 
surface,  especially  during  the  critical  final  150  feet  of 
descent.  For  the  pilot  on  the  left  side,  the  movement  of 
particles  from  right  to  left  created  a  vection  sensation  of 
rightward  movement.  The  normal  response  is  to  correct 
by  application  of  a  left  control  input.  During  the  final  100 
feet  Buzz  Aldrin  frequently  called  “left  drift”  from 
instrumentation  readings  while  Neil  Armstrong  did  his 
best  to  correct.  His  comments  concerning  the  last  portion 
of  descent  were,  “I  think  I  was  probably  over  controlling 
a  little  bit  in  lateral.  I  was  confused  somewhat  in  that  I 
couldn't  really  determine  what  my  lateral  velocities  were 
due  to  the  dust  obscuration  of  the  surface  ”  and  "I  was 
surprised  that  I  had  as  much  trouble  as  I  did  in 
determining  translational  velocities.  I  don't  think  I  did  a 
very ;  good  job  of flying  the  vehicle  smoothly  in  that  period 
of  time.  I  felt  I  was  a  little  bit  erratic. "  (Jones,  1995). 
When  the  lunar  lander  touched  down  it  was  still  drifting 
to  the  left  but  fortunately  the  velocity  was  not  sufficient  to 
turn  over  the  lander  module. 

Illusions  produced  by  vection  can  be  countered  or 
reduced  as  this  set  of  pilot  experiments  demonstrate. 

When  pilots  or  astronauts  have  the  availability  of  tactile 
velocity  cueing,  the  opportunity  exists  to  prevent  mishaps 
associated  with  vection  illusions  in  the  aviation  and  space 
environments.  Tactile  technologies  being  developed  by 
the  Army  will  be  available  when  astronauts  return  to  the 
lunar  surface. 

Motion  sickness,  including  simulator  sickness,  is 
influenced  by  many  factors  other  than  physical  motion. 
The  ameliorative  effects  of  accurate  visual  stimuli  on  sea 
sickness  are  well  known  and  used  by  seasick-prone 
individuals  when  they  leave  the  hold  to  stand  on  deck 
where  they  can  view  the  horizon.  It  is  postulated  that 
veridical  visual  motion  is  integrated  with  other  sensory 
information  to  reduce  the  effects  of  the  motion  stimulus. 
While  preparing  the  TSAS  for  the  UH-60  hover 
demonstration,  several  tactile  algorithms  were  developed 


to  provide  a  sense  of  vertical  flow  or  motion  for  the 
helicopter  pilot.  One  such  algorithm  sequentially 
activated  the  eight  rows  in  a  torso  display  which  provided 
a  sense  of  moving  up  or  down  depending  on  the  direction 
of  row  activation.  Using  touch,  in  a  similar  manner  as 
vision  is  used  to  provide  veridical  motion  sensations;  it 
may  be  possible  to  offset  the  effects  of  nauseogenic 
motion  stimuli  through  application  of  tactile  flow  cues. 

Further  research  using  more  accurate  eye  movement 
recording  devices  is  needed  to  understand  the  basis  of 
inter-subject  differences.  The  source  of  individual 
differences  in  perception  is  often  difficult  to  examine 
experimentally  since  there  are  so  many  factors  involved, 
one  of  which  is  the  influence  of  past  experience  including 
training.  In  this  experiment,  which  was  essentially  a  pilot 
series  to  determine  whether  tactile  flow  stimuli  could  alter 
visual  illusions,  there  was  not  an  opportunity  to  examine 
the  role  of  training.  It  is  anticipated  that  one  or  more 
training  sessions  for  subjects  to  correlate  the  tactile  with 
visual  and  motion  experiences  will  result  in  more  uniform 
reflex  responses.  This  is  a  critical  area  for  future  studies 
and  may  explain  the  variation  in  eye  movement 
recordings  between  individuals. 

5.  CONCLUSIONS 

This  experiment  demonstrates  that  tactile  stimuli  can 
influence  both  the  perceptual  and  reflex  processes 
involved  in  the  visual-vestibular-somatosensory 
integration  responsible  for  self-motion  perception  in 
space.  The  opportunity  to  enhance  the  level  of  immersion 
for  personnel  using  simulators  is  only  one  facet  of  tactile 
technology.  The  proprioceptive  system  is  comprised  of 
the  skin-muscle-joint  and  vestibular  systems.  Depending 
on  the  degree  to  which  the  tactile  cues  can  influence 
vection,  it  may  be  possible  to  substitute  tactile 
proprioceptive  motion  cues  in  lieu  of  vestibular  cues 
provided  through  the  motion-base  currently  used  in  high 
fidelity  simulators.  By  eliminating  the  need  for  motion, 
simulators  can  become  significantly  cheaper  and  more 
portable  for  personnel  who  are  in  need  of  mission 
simulators  close  to  the  site  of  deployment.  Since  tactile 
cues  can  change  motion  perceptions,  it  may  be  possible  to 
effect  a  reduction  in  simulator  sickness  for  both  motion 
and  non-motion  based  simulators.  Tactile  stimuli  can 
provide  motion  and  position  cues  to  operators  of 
unmanned  systems  as  well  as  enhancing  the  situation 
awareness  of  personnel  in  vehicles  which  provide 
restricted  or  no  visual  reference  to  the  outside 
environment. 
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